Deep reinforcement learning for recursive segmentation

ABSTRACT

Systems and methods are provided for generating segmented output from input regardless of the resolution of the input. A single trained network is used to provide segmentation for an input regardless of a resolution of the input. The network is recursively trained to learn over large variations in the input data including variations in resolution. During training, the network refines its prediction iteratively in order to produce a fast and accurate segmentation that is robust across resolution differences that are produced by MR protocol variations.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/643,348, filed Mar. 15, 2018, which is hereby incorporated byreference in its entirety.

FIELD

The present embodiments relate to medical image processing.

BACKGROUND

The rapid development of noninvasive brain imaging technologies hasopened new horizons in analyzing and studying the anatomy and functionof the body. In an example, progress in accessing brain injury andexploring brain anatomy has been made using magnetic resonance (MR)imaging. The advances in brain MR imaging have also provided data withan increasingly high level of quality. The analysis of the MR datasetshas become a tedious and complex task for clinicians, who have tomanually extract important information. This manual analysis is oftentime-consuming and prone to errors. More recently, computerized methodsfor MR image segmentation, registration, and visualization have beenextensively used to assist doctors and clinicians in qualitativediagnosis.

The medical imaging environment is highly diverse in terms of dataacquisition, contrast or resolution. Brain Segmentation for instance, isa standard preprocessing step for neuroimaging applications, often usedas a prerequisite for anomaly detection, tissue segmentation, andmorphometry applications. Brain MR segmentation is an essential task inmany clinical applications because it influences the outcome of theentire analysis. This is because different processing steps rely onaccurate segmentation of anatomical regions. For example, MRsegmentation is commonly used for measuring and visualizing differentbrain structures, for delineating lesions, for analyzing braindevelopment, and for image-guided interventions and surgical planning.Each clinical application may require or use different resolutions ordimensions. To perform the segmentation task, each clinical applicationmay thus require a dedicated segmentation application or network.

Automating brain segmentation is challenging due to the sheer amount ofvariations of brain shapes and sizes as well as variation in imaging.Protocol differences in MR acquisition may lead to variations in imageresolution. In an example, a first medical imaging scan may use a firstresolution while a second medical imaging scan may use a secondresolution. The resolutions or dimensions of the resulting images orvolumes may be different due to the intended use of the imaging, makingautomated brain segmentation less reliable.

SUMMARY

By way of introduction, the preferred embodiments described belowinclude methods and systems for generating and applying a single trainednetwork for segmentation of MR data regardless of the resolution of theMR data. The network is trained reclusively using machine learningtechniques. The resolution of the input to the network is adjusted up ordown. Feature maps generated by the network are applied to the inputs ofthe next iteration. The network is trained to be robust given differentresolution input MR images or volumes.

In a first aspect, a method is provided for generating segmentedmagnetic resonance volumes in a magnetic resonance imaging system. Apatient is scanned by the magnetic resonance imaging system; magneticresonance volume data resulting from the scanning. The magneticresonance volume data is input to a trained network that is recursivelytrained to generate segmented volumes from input magnetic resonancevolume data regardless of a resolution of the input magnetic resonancevolume data. The trained network generates a segmented magneticresonance volume from the input magnetic resonance volume data. Thesegmented magnetic resonance volume is displayed.

In a second aspect, a method is provided for training a network togenerate segmented magnetic resonance images regardless of inputresolution. An MR image of a plurality of MR images of a set of trainingdata is input into a network using a first quantity of input channels.The quantity is equal to the quantity of classifications provided by thenetwork. The network generates a segmented image including the quantityof probability maps. The segmented image is compared to a ground truthsegmented image for the MR image. The network is adjusted based on thecomparison. A reinforcement agent selects a resolution action as afunction of the comparison. Each of the first quantity of input channelsof the MR image are multiplied by a respective probability map of thequantity of probability maps. The resolution action is performed on theMR images for each of the quantity of input channels. The altered MRimages of the quantity of input channels are input into the network.Generating, comparing, adjusting, selecting, multiplying, performing,and inputting are repeated for at least five iterations. The trainednetwork is output.

In a third aspect, a system is provided generating a trained networkconfigured to use inputs of different resolutions. The system includes amagnetic resonance imaging system, a memory, and an image processor. Themagnetic resonance imaging system is configured to acquire magneticresonance data at different resolutions.

The memory is configured to store the magnetic resonance data,associated labeled magnetic resonance data, and the network. The imageprocessor is configured to recursively train the network using an inputvolume from the magnetic resonance data and an associated labeled volumeusing back propagation. The network generates probability maps from aSoftMax activation layer at each iteration of the recursive training.The input volume is up sampled or pooled to different resolutions foreach iteration. After each iteration, the probability maps aremultiplied with the input volume and input back to the network. Therecursive training is repeated for each volume in the magnetic resonancedata.

The present invention is defined by the following claims, and nothing inthis section should be taken as a limitation on those claims. Furtheraspects and advantages of the invention are discussed below inconjunction with the preferred embodiments and may be later claimedindependently or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee. The components and the figures are not necessarilyto scale, emphasis instead being placed upon illustrating the principlesof the invention. Moreover, in the figures, like reference numeralsdesignate corresponding parts throughout the different views.

FIG. 1 depicts an example MR system.

FIG. 2 depicts an example workflow for training and applying a trainednetwork.

FIG. 3 depicts an example workflow for recursively training a network.

FIGS. 4A and 4B depict example diagrams of two states of the recursivetraining of FIG. 3.

FIG. 5 depicts an example diagram for recursively training a network.

FIG. 6 depicts an example flowchart for applying a trained network.

FIG. 7 depicts two example of output segmented data from a network.

FIG. 8 depicts a system for training and applying a recursively trainednetwork.

DETAILED DESCRIPTION

A single trained network is used to provide segmentation for an inputregardless of a resolution of the input. A network is recursivelytrained using machine learning techniques to learn over large variationsin the input data including variations in resolution. During training,the network refines its prediction iteratively in order to produce afast and accurate segmentation that is robust across resolutiondifferences that are produced by MR protocol variations.

One field where segmentation is important is in the field of medicalimaging. In image processing, segmentation is the process of dividing aninput into different parts or sections. One field that values accuratesegmentation is MR image processing. Methods for initial MR analysisfall into two general categories: classification and segmentation.Classification assigns a label to an MR series, e.g. normal or abnormal,level of severity, or a diagnosis. Segmentation is the process ofdelineating the boundaries, or contours, of various tissues. Imagesegmentation may be performed on two dimensional images, sequences oftwo-dimensional images, three-dimensional volume, or sequences ofthree-dimensional volumes. If the data is defined in three-dimensionalspace (e.g., obtained from a series of MR images), each image slice maybe segmented individually in a slice-by-slice manner. Thetwo-dimensional slices are then connected into a 3D volume or acontinuous surface. Alternatively, the segmentation is of the volumerather than slice-by-slice.

FIG. 1 depicts an MR system 100 for acquisition of frequency domaincomponents representing MR image data for storage in a storage array.The MR system 100 includes a control unit 20 configured to process theMR signals and generate images of the body for display to an operator.The control unit 20 may store the MR signals and images in a memory 24for later processing or viewing. The control unit 20 may include adisplay 26 for presentation of images to an operator. The MR scanningsystem 100 is only exemplary, and a variety of MR scanning systems maybe used to collect the MR data.

In the MR system 100, magnetic coils 12 create a static base or mainmagnetic field in the body of patient 11 or an object positioned on atable and imaged. Within the magnet system are gradient coils 14 forproducing position dependent magnetic field gradients superimposed onthe static magnetic field. Gradient coils 14, in response to gradientsignals supplied thereto by a gradient and control unit 20, produceposition dependent and shimmed magnetic field gradients in threeorthogonal directions and generate magnetic field pulse sequences. Theshimmed gradients compensate for inhomogeneity and variability in an MRimaging device magnetic field resulting from patient anatomicalvariation and other sources.

The control unit 20 may include a RF (radio frequency) module thatprovides RF pulse signals to RF coil 18. The RF coil 18 producesmagnetic field pulses that rotate the spins of the protons in the imagedbody of the patient 11 by ninety degrees or by one hundred and eightydegrees for so-called “spin echo” imaging, or by angles less than orequal to 90 degrees for “gradient echo” imaging. Gradient and shim coilcontrol modules in conjunction with RF module, as directed by controlunit 20, control slice-selection, phase-encoding, readout gradientmagnetic fields, radio frequency transmission, and magnetic resonancesignal detection, to acquire magnetic resonance signals representingplanar slices of the patient 11.

In response to applied RF pulse signals, the RF coil 18 receives MRsignals, e.g. signals from the excited protons within the body as theprotons return to an equilibrium position established by the static andgradient magnetic fields. The MR signals are detected and processed by adetector within RF module and the control unit 20 to provide an MRdataset to an image data processor 22 for processing into an image. Insome embodiments, the image data processor 22 is located in the controlunit 20, in other embodiments, the image data processor 22 is locatedremotely. A two or three-dimensional k-space storage array of individualdata elements in a memory 24 of the control unit 20 stores correspondingindividual frequency components including an MR dataset. The k-spacearray of individual data elements includes a designated center, andindividual data elements individually include a radius to the designatedcenter.

A magnetic field generator (including coils 12, 14 and 18) generates amagnetic field for use in acquiring multiple individual frequencycomponents corresponding to individual data elements in the storagearray. The individual frequency components are successively acquiredusing a Cartesian acquisition strategy as the multiple individualfrequency components are sequentially acquired during acquisition of anMR dataset. A storage processor in the control unit 20 stores individualfrequency components acquired using the magnetic field in correspondingindividual data elements in the array. The row and/or column ofcorresponding individual data elements alternately increases anddecreases as multiple sequential individual frequency components areacquired. The magnetic field acquires individual frequency components inan order corresponding to a sequence of substantially adjacentindividual data elements in the array, and magnetic field gradientchange between successively acquired frequency components issubstantially minimized.

One use of MR imaging is in studying a patient's brain. Steps foranalysis generally include the classification of acquired MR data intospecific tissue types and the identification and description of specificanatomical structures. Classification may assign to each element in theimage a tissue class when the classes are defined in advance. In thecase of brain MR, for tissue classification, image elements may beclassified into three main tissue types: white matter (WM), gray matter(GM), and cerebrospinal fluid (CSF). Classification of the tissue typesrequires segmentation of the MR data into different parts. Thesegmentation results may also be used in different applications such asfor analyzing anatomical structures, for studying pathological regions,for surgical planning, and for visualization.

In addition to classification of the brain material, segmentation may beused to preprocess MR data so that further analysis or segmentation maybe performed. MR brain scan data generally includes some non-braintissues such as skin, fat, muscle, neck, and eye balls. The presence ofthe non-brain tissues is an obstacle for further automatic brain imagesegmentation and analysis. A preliminary preprocessing step may berequired to isolate the brain from extra-cranial or non-brain tissues.The preliminary preprocessing step is commonly referred to as skullstripping. Brain images that are preprocessed with skull strippingtypically lead to better segmentation and classification of differentbrain regions that results in better and more accurate diagnosis ofvarious brain-related diseases. Skull stripping may also be used as apreprocessing step prior to other image processing algorithms such asimage registration and warping, brain volumetric measurement,inhomogeneity correction, tissue classification, analysis of corticalstructure, cortical surface reconstruction, cortical thicknessestimation, identification of brain parts, multiple sclerosis analysis,Alzheimer's disease, schizophrenia, and monitoring the development oraging of the brain among other uses.

One issue with both segmentation and skull stripping is the inputs varyin resolution and dimensions. MR image acquisition is highly diverse interms of data acquisition, contrast or resolution. For skull stripping,in particular, the end uses may vary and as such use different sizedimages or volumes as inputs. Different protocols and machines may beused to acquire the MR data depending on the intended use. In anexample, a first protocol may use a resolution of 256×256×256 while asecond protocol may use a resolution of 128×128×32. Classical supervisedML does not adequately handle this real-world data problem. Eachdifferent resolution may require a separate network trained exclusivelyon training data that is the appropriate resolution. As there arenumerous different resolutions, this may require numerous networks andtraining sessions. This also may require additional, difficult to locatein sufficient numbers, ground truth data to train the numerous networks.

Embodiments provide a supervised deep machine learning (ML) approach totrain a single network to be robust across volume resolution whileproviding segmentation. Embodiments provide a combined method ofsupervised deep learning, deep reinforcement learning and recursivelearning to provide a single trained network that may be used for inputsof varying resolution. The disclosed trained network may be implementedto computationally facilitate processing of medical imaging data andconsequently improving and optimizing medical diagnostics. By using asingle network that is robust across multiple resolutions, errors arediminished and outcomes are improved. The use of a single network isefficient in that fewer resources are used to both train and store asingle network as opposed to multiple networks for multiple resolutions.The use of a single network further limits errors by removing aselection step and allowing clinicians or physicians to use a singlepathway from MR imaging to analysis for the given application (e.g.,brain scan).

FIG. 2 depicts an example flowchart for providing resolution independentimage processing for a magnetic resonance imaging system. The flowchartincludes two stages, a training stage 151 for generating or training thenetwork using a collection of training data (labeled data) and anapplication stage 150 for applying the generated/trained network to newunseen (unlabeled) data. The training stage 151 includes acquiring 101training data and inputting the training data into a network in order togenerate 103 a trained network. The output is a trained network that isapplied 153 in the application stage 150. The application stage 150includes acquiring 151 unseen MR data, applying 153 the trained networkthat was trained during the training stage 151, and outputting 157segmented data.

The training stage 151 and application stages 150 are described indetail below at FIGS. 3 and 6, respectively. The training stage 151 maybe performed at any point prior to the application stage. The trainingstage 151 may be repeated after new training data is acquired. Theapplication stage 150 may be performed at any point after the trainingstage 151 generates the trained network and MR data is acquired. Theapplication stage 150 may be performed, for example, during (e.g. realtime) or directly after a medical procedure is performed or as part ofplanning for a particular patient. Alternatively, the application stage150 may be performed at a later point using MR data acquired from animaging scan and stored, for example, in a PACS.

The embodiments below use brain segmentation (skull stripping) as anexample of the application of the trained network. The trained networkmay be applied to any segmentation problem found in medical imaging orother fields when provided with appropriate training data. The trainednetwork provides accurate segmentation regardless of the resolution ofthe input image or volume.

FIG. 3 depicts an example flowchart for generating a resolution robusttrained network 305 using machine learning. The trained network 305 maybe a machine learned network 305 or machine trained network 305. Thetrained network 305 is trained using a supervised training method.Unlabeled training data 201 is input into the network 205 that generatesan outcome that is compared against associated labeled training data201. Using backpropagation and a gradient, the network 205 adjustsinternal parameters based on the comparison. The process is repeateduntil the network 205 may no longer be improved or a set point isreached. The training process for the network 205 may include bothrecursive learning 221 and reinforcement learning 223.

The acts are performed by the system of FIG. 1, FIG. 3, FIG. 8, othersystems, a workstation, a computer, and/or a server. Additional,different, or fewer acts may be provided. The acts are performed in theorder shown (e.g., top to bottom) or other orders.

At 201, training data 201 is acquired. Training data 201 may includeground truth data or gold standard data. Ground truth data and goldstandard data is data that includes correct or reasonably accuratelabels. For the segmentation problem, the training data 201 includes theoriginal data and associated segmented data. Labels for segmentationpurposes include labels for each voxel in the segmented data, anoutline, or a fit shape. The segmented data may be generated and labeledusing any method or process, for example, manually by an operator orautomatically by one or more automatic methods. Different training data201 may be acquired for different segmentation tasks. For example, afirst set of training data 201 may be used to train a first network 205for segmenting brain data, while a second set of training data 201 maybe used to train a second network 205 for segmenting heart data. Thetraining data 201 may be acquired at any point prior to inputting thetraining data 201 into the network 205. The training data 201 mayinclude volumes of different resolutions or contrast. The training data201 may be updated after acquiring new data. The updated training data201 may be used to retrain or update the network 205.

In an embodiment, the training data 201 is MR data. As used herein, MRdata includes both raw MR data and processed MR data. Processed MR datamay include image and volume data. MR data may include 2D images,sequences of 2D images, 3D volumetric imagery, or sequence of 3Dvolumetric imagery. If the MR data is defined in 3D space (e.g.,obtained from a series of MR images), each image “slice” may be providedindividually in a “slice-by-slice” manner. Alternatively, the MR datamay be provided as 3D volumetric data directly to the network 205. Theexamples described herein use three-dimensional MR data referred to asvolumes. Volumes are encoded using an array of elements referred to asvoxels. A voxel represents a value on a regular grid inthree-dimensional space. The methods and systems described below mayalso be used with two-dimensional MR data referred to as images. Animage is encoded using a bitmap of pixels.

At 203, a volume from the training data 201 is input to the network 205using a first number of input channels. For an initial state, the sameMR volume data may be used for a plurality of input channels to thenetwork 205. For subsequent iterations, the MR volume data is multipliedby the output probability maps 207 of the network 205. The number ofinput channels is equal to the number of classes provided by the network205. As an example, if the network 205 is configured to generate twoclasses (e.g. brain and non-brain material), the network 205 is thenconfigured to use two input channels. A first input channel ismultiplied by the brain probability map and the second input channel ismultiplied by the non-brain probability map.

The initial MR volume data (for an initial state) or the adjusted MRvolume data (for subsequent states) is input into the network 205. Thenetwork 205 is configured to segment the input volume data. Segmentationseparates different portions from one another. In the example of skullstripping, non-brain tissues such as fat, skull, or neck may includeintensities that overlap with intensities of brain tissues. The braintissue may be identified before further processing may be performed.Segmentation for skull stripping classifies voxels as brain or nonbrain.The result may either be a new volume with just brain voxels or a mask,that includes, for example, a value of 1 for brain voxels and 0 for therest of tissues. In general, the brain voxels include GM, WM, and CSF ofthe cerebral cortex and subcortical structures, including the brain stemand cerebellum. The scalp, dura matter, fat, skin, muscles, eyes, andbones are classified as nonbrain voxels.

The network 205 generates probability maps 207 for each of the classes.A probability map 207 may include a value that represents an estimationby the network 205 whether or not each voxel represents the respectiveclass. The probability maps 207 may be generated using a SoftMaxactivation layer at the output of the network 205. The SoftMaxactivation layer takes an un-normalized vector from the network 205, andnormalizes the vector into a probability distribution. The probabilitymaps 207 that are generated by the network 205 are used to generate theoutput segmented volume. Combined, the probability maps 207 represent aprobability for each of the classes for each voxel. For each voxel, themost probable class may be selected and the voxel thus assigned to theclass. In an example, for a two-class segmentation, there are twoprobability maps 207, that when combined provide probability for each ofthe voxels to be one or the other class. The output data may bevisualized as a two-colored volume where each voxel is colored with therespective class for which it is assigned the highest probability.

The output segmented data 209 is compared against the ground truth ofthe training data 201 to determine a score 211. The score 211 mayrepresent the level of differences between the output segmented data 209and the correct segmented data (ground truth or gold standard) providedwith the training data 201. The score 211 is used to adjust weights ofthe network 205 using backpropagation and a gradient.

The score 211 may be calculated as a dice score that is calculated as:

$\frac{{P\bigcap T}}{\left( {{P} + {T}} \right)/2}$where P represents the segmented area and T represents the ground trutharea. Dice scores range from 0 to 1, where a score of 1 representsperfect segmentation. Other scores, errors, or loss functions may beused.

In an embodiment, the network 205 may be configured as a DenseNet. TheDenseNet connects each layer to every other layer in a feed-forwardfashion. For each layer in the DenseNet, the feature-maps of allpreceding layers are used as inputs, and the output feature-map of thatlayer is used as input into all subsequent layers. In the DenseNet, foreach layer, the feature maps of all preceding layers are used as inputs,and its own feature maps are used as inputs into all subsequent layers.To reduce the size of the network, the DenseNet may include transitionlayers. The layers include convolution followed by average pooling. Thetransition layers reduce height and width dimensions but leave thefeature dimension the same. The final layer may be a SoftMax activationlayer that generates the probability maps 207 used in generating aninput for a subsequent iteration. The ML generator network may furtherbe configured as a U-net. The U-Net is an encoder-decoder combination inwhich the outputs from the encoder-half of the network are concatenatedwith the mirrored counterparts in the decoder-half of the network. Skipconnections between the encoder and decoder at any level of resolutiongreater than the bottleneck may be used.

Other network configurations may be used, such as deep architecturesincluding convolutional neural network (CNN) or deep belief nets (DBN).CNN learns feed-forward mapping functions while DBN learns a generativemodel of data. In addition, CNN uses shared weights for all localregions while DBN is a fully connected network (e.g., includingdifferent weights for all regions of an image). The training of CNN isentirely discriminative through back-propagation. DBN, on the otherhand, employs the layer-wise unsupervised training (e.g., pre-training)followed by the discriminative refinement with back-propagation ifnecessary. In an embodiment, the arrangement of the network 205 is afully convolutional network (FCN). Alternative network arrangements maybe used, for example, a 3D Very Deep Convolutional Networks (3D-VGGNet).VGGNet stacks many layer blocks containing narrow convolutional layersfollowed by max pooling layers. A 3D Deep Residual Networks (3D-ResNet)architecture may be used. A Resnet uses residual blocks and skipconnections to learn residual mapping.

For each network configuration, rather than pre-programming the featuresand trying to relate the features to attributes, the deep architectureof the network is defined to learn the features at different levels ofabstraction based on an input data with or without pre-processing. Thefeatures are learned to reconstruct lower level features (i.e., featuresat a more abstract or compressed level). For example, features forreconstructing an image are learned. For a next unit, features forreconstructing the features of the previous unit are learned, providingmore abstraction. Each node of the unit represents a feature. Differentunits are provided for learning different features.

In an embodiment, the network 205 is a three-dimensional image-to-imagenetwork. The network 205 includes twenty-four initial feature maps, agrowth rate of twenty-four and two levels of pooling. The network 205produces a probability output from a SoftMax activation layer at eachstep of the recurrent process. The probability output maps 207 obtainedat a given step are used as context information for a subsequentiteration. The network 205 includes as many input channels as outputchannels, e.g. one per class of the segmentation problem. In an example,there are two output channels and two input channels. A first outputchannel may correspond to the brain and the second one for non-braindata. At an initial step, a volume is provided on each input channel.After each process iteration, probability output maps are multipliedwith the input data and input back to the network for the nextiteration. Each volume provided on the different input channels is thusa subset of the full input volume, that can be summed up to obtain theoriginal full volume.

FIGS. 4A and 4B depicts an initial state (T) and a subsequent state(T+1) in the iterative process. Both FIGS. 4A and 4B include an input203, the network 205, and the output probability maps 207. For theinitial state, an MR volume 203 from a set of training data 201 is inputinto the network 205. At each state T, the network 205 takes new inputs203 that will be related to the previous time step T−1. The network 205includes as many input channels (N) as there are classes in the problem.At state T, each channel will take the input X multiplied by theprobability map 207 of each segmented class obtained at step T−1 (theinput is only X at T=0). FIG. 4A depicts an initial state (T) where theonly inputs are the original inputs 203. FIG. 4B depicts a state (T+1)where the input is the original input multiplied by the probability mapof the previous state (T).

Referring back to FIG. 3, the generated score is also used by thereinforcement agent 215. For each iteration, the input volume 203 mayalso be up sampled, down sampled, or pooled to change the resolution.The decision of whether to up sample, down sample, or pool theresolution may be performed by a reinforcement agent 215 or may selectedrandomly. The reinforcement agent 215 generates a resolution action 217to be performed on the volume to be input to the network 205 for asubsequent iteration. The action is whether to work at lower, higher orsame resolution (e.g. apply a pooling or an up-sampling action to thenext input volume). The reinforcement agent 215 may use reinforcementlearning to select the resolution action 217. Reinforcement learningprovides a mechanism to identify immediate solutions given a long-termgoal. Reinforcement learning may be described using the concepts ofagents, environments, states, actions and rewards. The reinforcementagent 215 is an agent that takes actions, e.g. selects the resolutionaction 217 of how to change the resolution for the next state of thetraining process. The agent may decide to up sample, pool, down sample,or take no action. The magnitude of up sampling and down sampling mayalso be selected by the agent. The state of the iterative process is thecurrent situation or current environment that the agent is acting on. Inan example, the state may be the difference between the output 209 andthe input data 203 of the network 205. The agent 215 will output anaction 217 to take on the combination of the output 209 and the inputdata 203, and feed the segmentation network for the new step. A rewardis the feedback by which the process measures the success or failure ofan agent's actions. From any given state, an agent sends output in theform of actions to the environment, and the environment returns theagent's new state (that resulted from acting on the previous state) aswell as rewards, if there are any. Rewards may be immediate or delayed.Rewards effectively evaluate the agent's action. A reward may becalculated from a DICE score 211 in a final resolution obtained from thesegmentation. A policy may be provided to guide the strategy that theagent employs to determine the next action based on the current state.It maps states to actions, the actions that promise the highest reward.

The reinforcement agent 215 identifies the environment based on theoutput of the network 205. The reinforcement agent 215 selects aresolution action 217 for how to adjust the resolution of the next inputas function of a defined policy. The result of the resolution action 217is measured by a score 211 of a final resolution of the segmentation. Areward is provided for the resolution action 217 that providesadditional feedback for the agent to identify future resolution actions217.

For each iteration, the resolution action 217 is performed on the MRvolume data in addition to the multiplication by the probability maps207. The change in resolution to the input provides a mechanism for thenetwork 205 to learn how to generate accurate segmentation regardless ofthe resolution of the input volume.

The process is repeated for a number of iterations. A predefined numberof iterations may be performed for each input volume from the trainingdata 201. In an example, five, ten, twenty, or more iterations may beused. Each iteration takes the output probability maps 207 and the inputvolume and multiplies the output probability maps 207 and copies of theinput volumes together to generate the input channels for the currentstate. The number of iterations may be selected as a tradeoff betweenperformance and time constraints. More iterations may produce a moreaccurate network 205 but may take long to train.

After the iterative process is finished, another volume is provided fromthe training data 201. The recursive process is then performed on thenew volume. This is repeated until all volumes in the training data 201set have been processed or the network 205 is determined to be trained.

In an embodiment, the network 205 may also be hardened against thedifferent data distribution problem that arises due to contrastvariations. Data normalization may be performed prior to inputting thetraining data 201 into the network 205. Normalization may includenormalizing all the input volumes 203 to the same dimensions.Additionally, or alternatively, in addition to segmenting the input data203, the network 205 may output an image or volume that is used as inputat a subsequent step t+1 instead of reusing the original image orvolume. For the process to converge properly, multiple steps areperformed before updating the network 205, as the network 205 learns theinput recursively. The model normalizes its input over iterations, toextract necessary features for the final segmentation. FIG. 5 depicts anexample of using an output image as the input image for a subsequentiteration. Similar to FIG. 4B, FIG. 5 includes the network 205 and theprobability maps 207. The input to the network 205, however, is not theoriginal volume from the training data set, but rather a volume 233generated at the last iteration (T−1) by the network. The network 205inputs the volume 233 and output another volume that will be used forthe input at (T+1). By reinjecting a predicted input through the inputto the network may be automatically normalized.

Once the network 205 is trained, the trained network 305 may be applied.The trained network 305 with defined or learnt features is used toextract features from previously unseen input MR data. The trainednetwork 305 extracts values for features from the acquired MR data withor without other information to generate a segmented image.

FIG. 6 depicts an example workflow for generating segmented medical datafor uses in medical diagnosis, prognosis, planning, treatment, oranalysis. The acts are performed by the system of FIG. 1, FIG. 5, FIG.8, other systems, a workstation, a computer, and/or a server.Additional, different, or fewer acts may be provided. The trainednetwork 305 may be used for any segmentation task. The examples belowuse medical imaging data and more specifically skull stripping, butother types of image data may be segmented. The examples usethree-dimensional volumes but the acts may be applied to two-dimensionalimages. The acts are performed in the order shown (e.g., top to bottom)or other orders.

At 301, an object is scanned by the magnetic resonance imaging system toacquire MR data 301. As depicted and described in FIG. 1 above, the MRdata 301 may be acquired using an MR scanner. For example, gradientcoils, a whole-body coil, and/or local coils generate a pulse or scansequence in a magnetic field created by a main magnet or coil. Thewhole-body coil or local coils receive signals responsive to there-orientation of molecules shifted due to the scan sequence. In anembodiment and used as an example below, the MR data may represent imagedata for a brain of a patient. Different objects, organs, or regions ofa patient may also be scanned.

The MR data 301 is k-space data or image data. Image data may be MR data301 after Fourier transform into object space. The image data may be atany point after transform, so may be scalar values or may be formattedas RGB values for a display screen. The MR data 301 may be scan data tobe used to generate an image on a display. The acquired MR data 301 maybe data being processed to generate an image, data formatted fordisplay, or data that has been used to display. The MR data 301 may bedata with no or some image processing.

In an embodiment and used as an example below, the MR data 301 mayrepresent a volume. A three-dimensional dataset is obtained. As k-spacedata, information content may be provided that is responsive to athree-dimensional distribution of locations, but the data itself doesnot directly represent the locations prior to transform. In alternativeembodiments, a two-dimensional dataset representing or responsive totissue in a plane is obtained. In other embodiments, sequences of MRdata responsive to the same tissue over time are acquired for training.

Alternative methods may be used to acquire the MR data 301. The MR data301 may be acquired remotely from the server or workstation or may beacquired at a different time, for example, hours or days prior to theprocessing provided below. The MR data may be stored locally onsite oroffsite, for example in the cloud.

The MR data 301 may be acquired at different resolutions. For example,one set of MR data may be 256×256×256 while another may be 128×128×32.The MR data 301 may be normalized to a standard dimension. The trainednetwork 305 may be configured to input a standard dimensional image orvolume. The MR data 301 may be converted to the standard dimension. If,for example, the MR data 301 is too dimensionally small, the MR data 301may be up sampled to the standard dimensions.

At 303, the MR data 301 is input into a trained network 305. The trainednetwork 305 is configured to input multiple different resolutions of MRIdata 301 that result from different types of protocols, sequences, orscans. Rather than pre-programming the features and trying to relate thefeatures to attributes, the deep architecture of the network is definedto apply the features at different levels of abstraction based on aninput image data with or without pre-processing. The features werelearned to reconstruct lower level features (i.e., features at a moreabstract or compressed level). For example, features for reconstructingan image were learned. For a next unit, features for reconstructing thefeatures of the previous unit were learned, providing more abstraction.Each node of the unit represents a feature. Different units are providedfor learning different features.

The trained network 305 may be a dense image-to-image network trained togenerate a segmented image given MR data. The network 205 is trainedoffline, e.g. prior to the acts of FIG. 6. The training process includesrecursively training the network 205 using different resolutions. Thedifferent resolutions are selected using a reinforcement agent 215. Thenetwork 205 is trained recursively by using an output of the network 205in combination with a change in resolution as the input to a subsequentiteration. During the recursive process, the network 205 is adjustedusing back propagation in order to generate a more accurate segmentedoutput. The recursive process for an input volume ends after a number ofiterations. The process is repeated for each volume in the training data201. The training process produces a trained network 305 that isconfigured to generate accurate segmentations regardless of theresolution of the input volume.

The trained network 305 generates segmented data from the input MR data209. The segmented data 209 may include boundaries for different typesof classes. For example, for skull stripping, the output image or volumemay include designations for brain tissue or non-brain tissue. FIG. 7depicts two examples of segmented data output 209 by the trained network305 and overlaid over the original images. In FIG. 7, the brain tissueis highlighted while the non-brain tissue is ignored. In another exampleof segmented data 209 for a brain, boundaries and classifications may beincluded for at least white matter, grey matter, and cerebrospinalfluid. Each pixel or voxel may be classified as a type of tissue. Theoutput image may identify the type of tissue depicted in the image.Different colors or shades of grey, for example, may be used to depictthe segmented image. The output image may be annotated with additionalinformation.

At 305, the trained network 305 outputs the segmented data 209. Theoutput segmented data 209 may be used for different procedures ordiagnosis. For example, the output segmented data may be displayed 313to an operator or physician. The output may be presented to an operatorwith labels or different colors representing different tissues or pointsof interest. The output may be two-dimensional or a rendering from athree-dimensional distribution. The output may be color or black andwhite. The image data and the segmented data may be stored 315 for lateruse.

In a skull stripping example and other examples, further processing maybe performed once the non-brain tissue has been removed from the images.The removal of the non-brain tissue allows the further processing oranalysis to proceed without having to deal with possible overlappingintensities between the brain and non-brain tissue resulting in fewercomputational resources being used, a shorter turnaround time, and moreaccurate results. The segmented data 209 may be used for medicalprocedures 307, medical diagnosis 309, or medical studies 311 before orafter additional processing.

FIG. 8 depicts one embodiment of a control unit for generating segmentedimages from MR data. The control unit includes an image processor 22, amemory 24, and a display 26. The control unit 20 may be connected with aserver 28 and an MR imaging device 36. Additional, different, or fewercomponents may be provided. For example, network connections orinterfaces may be provided, such as for networking between the controlunit 20 and server 28. A workstation with a user interface may beprovided for an operator to input data.

The MR imaging device 36 may be similar to the MR imaging device 36 asdepicted in FIG. 1. The MR imaging device 36 is configured to acquire MRdata that may be processed into one or more images or volumes by thecontrol unit 20. The control unit 20 may provide commands to the MRimaging device 36. Alternatively, the MR imaging device 36 may functionentirely on its own without any input from the control unit 20.

The image processor 22 (or processor) is a general processor, centralprocessing unit, control processor, graphics processor, digital signalprocessor, three-dimensional rendering processor, image processor,application specific integrated circuit, field programmable gate array,digital circuit, analog circuit, combinations thereof, or other nowknown or later developed device for processing an image. The processor22 is a single device or multiple devices operating in serial, parallel,or separately. The processor 22 may be a main processor of a computer,such as a laptop or desktop computer, or may be a processor for handlingsome tasks in a larger system, such as in the MR system. The processor22 is configured by instructions, design, hardware, and/or software toperform the acts discussed herein.

The server 28 may be co-located with the control unit 20 or may belocated remotely. The server 28 may connect to the MR system 100 orcontrol unit 20 via a network. The network is a local area, wide area,enterprise, another network, or combinations thereof. In one embodiment,the network is, at least in part, the Internet. Using TCP/IPcommunications, the network provides for communication between theprocessor 24 and the server 28. Any format for communications may beused. In other embodiments, dedicated or direct communication is used.

The server 28 may include the processor 24 or group of processors. Morethan one server 28 or control unit 20 may be provided. The server 28 isconfigured by hardware and/or software. In one embodiment, the server 28performs ML of the network 205. The server 28 may acquire and the memory24 may store MR data from multiple different MR systems.

The processor 24 and/or server 28 are configured to perform the actsdiscussed above for generating segmented images. The processor 24 and/orserver 28 may access and implement the code stored in memory 24.

The memory 24 may be a graphics processing memory, a video random accessmemory, a random-access memory, system memory, cache memory, hard drive,optical media, magnetic media, flash drive, buffer, database,combinations thereof, or other now known or later developed memorydevice for storing data or video information. The memory 24 is part ofthe control unit 20, part of a database, part of another system, apicture archival memory, or a standalone device. The memory 24 may storeimage data from the MR device 36. The memory 24 may store an instructionset or computer code configured to implement the network 205.

The memory 24 includes an instruction set or computer code forimplementing the network 205. In an embodiment, the memory 24 includes atrained network 305 and training data 201. In an embodiment, only thetrained network 305 is stored in memory 24. The trained network 305 maybe configured to input an MR volume and output a segmented MR volume.The trained network 305 may be configured to function regardless of theresolution of the input MR volume. To provide resolution independentsegmentation, a network 205 is trained recursively and with areinforcement agent 215. For an initial state, the network 205 takes asinput a volume or image from a training set of data. The network 205generates a plurality of probability maps 207. The number of probabilitymaps 207 matches the number of classes that also matches the number ofinput channels to the network 205. For subsequent states, the network205 takes as input the volume or image multiplied by the probabilitymaps 207 generated at the previous state. In addition, the volume orimage is up sampled or pooled to changes the resolution of the volume orimage. The decision to up sample or pool the volume or image may bedetermined using a reinforcement agent 215 trained using a reinforcementmechanism. The reinforcement mechanism may generate a reward for thereinforcement agent 215 based on a comparison score between the outputof the network 205 and ground truth/gold standard data.

The memory 24 or other memory is alternatively or additionally anon-transitory computer readable storage medium storing datarepresenting instructions executable by the programmed processor 22 forgenerating resolution independent segmented data. The instructions forimplementing the processes, methods and/or techniques discussed hereinare provided on non-transitory computer-readable storage media ormemories, such as a cache, buffer, RAM, removable media, hard drive, orother computer readable storage media. Non-transitory computer readablestorage media include various types of volatile and nonvolatile storagemedia. The functions, acts or tasks illustrated in the figures ordescribed herein are executed in response to one or more sets ofinstructions stored in or on computer readable storage media. Thefunctions, acts or tasks are independent of the particular type ofinstructions set, storage media, processor or processing strategy andmay be performed by software, hardware, integrated circuits, firmware,micro code, and the like, operating alone, or in combination. Likewise,processing strategies may include multiprocessing, multitasking,parallel processing, and the like.

The display 26 may be configured to display images to an operator. Thedisplay 26 may augment the images with additional information oroverlays. The display 26 may be configured to display the images in twodimensions, three dimensions, or, for example, in augmented or virtualreality scenarios.

In one embodiment, the instructions are stored on a removable mediadevice for reading by local or remote systems. In other embodiments, theinstructions are stored in a remote location for transfer through acomputer network or over telephone lines. In yet other embodiments, theinstructions are stored within a given computer, CPU, GPU, or system.

While the invention has been described above by reference to variousembodiments, it should be understood that many changes and modificationscan be made without departing from the scope of the invention. It istherefore intended that the foregoing detailed description be regardedas illustrative rather than limiting, and that it be understood that itis the following claims, including all equivalents, that are intended todefine the spirit and scope of this invention.

The invention claimed is:
 1. A method for generating segmented magneticresonance volumes in a magnetic resonance imaging system, the methodcomprising: scanning a patient by the magnetic resonance imaging system;magnetic resonance volume data resulting from the scanning; inputtingthe magnetic resonance volume data to a machine trained network that isrecursively trained using machine learning techniques to generatesegmented volumes from input magnetic resonance volume data regardlessof a resolution of the input magnetic resonance volume data, whereinrecursively training the network comprises generating probability mapsfrom a SoftMax activation layer at each iteration of the recursivetraining, up-sampling or pooling the input volume to differentresolutions for each iteration, and after each iteration multiplying theprobability maps with the up-sampled or pooled input volume andinputting the multiplied probability maps back to the network forsubsequent iteration; generating, by the machine trained network, asegmented magnetic resonance volume from the input magnetic resonancevolume data; and displaying the segmented magnetic resonance volume. 2.The method of claim 1, wherein the machine trained network is athree-dimensional image-to-image network.
 3. The method of claim 1,wherein the magnetic resonance imaging system is configured to acquiremagnetic resonance volume data of a brain of a patient.
 4. The method ofclaim 3, wherein the machine trained network is trained to perform asegmentation task comprising skull stripping of the magnetic resonancevolume data.
 5. The method of claim 4, further comprising: performing asubsequent segmentation of the segmented magnetic resonance volume toidentify tissue types.
 6. A method for training a network using machinelearning to generate segmented magnetic resonance images regardless ofinput resolution, the method comprising: inputting a magnetic resonanceimage of a plurality of magnetic resonance images of a set of trainingdata into a network using a quantity of input channels, wherein thequantity is equal to a quantity of classifications provided by thenetwork; generating, by the network, a segmented image including aquantity of probability maps equal to the quantity of classifications,wherein each probability map of the quantity of probability mapscomprises a probability map for a different class of the quantity ofclassifications; comparing the segmented image to a ground truthsegmented image for the magnetic resonance image; adjusting the networkbased on the comparison; selecting, by a reinforcement agent, aresolution action as a function of the comparison; multiplying each ofthe quantity of input channels of the magnetic resonance image by arespective probability map of the quantity of probability maps;performing the resolution action on the magnetic resonance images foreach of the quantity of input channels; inputting the altered magneticresonance images of the quantity of input channels into the network;repeating generating, comparing, adjusting, selecting, multiplying,performing, and inputting for at least five iterations; and outputting amachine trained network.
 7. The method of claim 6, wherein the networkis a dense image to image network.
 8. The method of claim 6, wherein theresolution action comprises one of up sampling, pooling, or no action.9. The method of claim 6, wherein the network is configured to input amagnetic resonance brain image and output a segmented magnetic resonancebrain image.
 10. The method of claim 6, wherein the reinforcement agentis trained using a reinforcement mechanism to select the resolutionaction.
 11. The method of claim 6, wherein the segmented image is usedas the input for a subsequent iteration.
 12. The method of claim 6,wherein generating, comparing, adjusting, selecting, multiplying,performing, and inputting is repeated for at least ten iterations. 13.The method of claim 6, wherein each magnetic resonance image of theplurality of magnetic resonance images of the set of training data isinput into the network prior to outputting the machine trained network.14. A system for generating a machine trained network configured to useinputs of different resolutions, the system comprising: a magneticresonance imaging system configured to acquire magnetic resonance dataat different resolutions; a memory configured to store the magneticresonance data, associated labeled magnetic resonance data, and anetwork; and an image processor configured to recursively train thenetwork using an input volume from the magnetic resonance data and anassociated labeled volume using back propagation; wherein the networkgenerates probability maps from a SoftMax activation layer at eachiteration of the recursive training; wherein the input volume is upsampled or pooled to different resolutions for each iteration; whereinafter each iteration, the probability maps are multiplied with the inputvolume and input back to the network; wherein the recursive training isrepeated for each volume in the magnetic resonance data.
 15. The systemof claim 14, wherein the network is configured to input a magneticresonance brain volume and output a segmented magnetic resonance brainvolume regardless of the resolution of the input magnetic resonancebrain volume.
 16. The system of claim 14, wherein the image processer isfurther configured to select whether the input volume is up sampled orpooled to different resolutions for each iteration as a function of aselection by a reinforcement agent stored in the memory.
 17. The systemof claim 16, wherein the reinforcement agent selects whether the inputvolume is up sampled or pooled to different resolutions for eachiteration as a function of a comparison between an output of the networkand an associated labeled volume at each iteration.
 18. The system ofclaim 14, wherein the network is a three dimensional dense image toimage network.
 19. The system of claim 14, wherein the image processoris configured to perform at least five iterations of the recursivetraining for each volume in the magnetic resonance data.